Deep classifiers have achieved great success in visual recognition. However, real-world data is long-tailed by nature, leading to the mismatch between training and testing distributions. In this paper, we show that the Softmax function, though used in most classification tasks, gives a biased gradient estimation under the long-tailed setup. This paper presents Balanced Softmax, an elegant unbiased extension of Softmax, to accommodate the label distribution shift between training and testing. Theoretically, we derive the generalization bound for multiclass Softmax regression and show our loss minimizes the bound. In addition, we introduce Balanced Meta-Softmax, applying a complementary Meta Sampler to estimate the optimal class sample rate and further improve long-tailed learning. In our experiments, we demonstrate that Balanced Meta-Softmax outperforms state-of-the-art long-tailed classification solutions on both visual recognition and instance segmentation tasks.

A careful implementation should be made for instance segmentation tasks. Firstly, we define f as, f ( x ) : = l (θ ) + t (24) where l ( θ ) and t is previously defined in the main paper.

Weaknesses: -The equations (3) and (4) are, however, very similar to [3] and [A, B] in the way that they force the minor-class examples to have larger decision values (i.e., \exp \eta_j) in training. The proposed softmax seems particularly similar to eq. (11) in [B]. The authors should have cited these papers and provided further discussion and comparison. This point limits the novelty/significance of the paper. It is hard for me to judge the novelty of the proposed meta sampler.

The paper first shows that the softmax gives a biased gradient estimation under the long-tailed setup, and proposes a balanced softmax to accommodate the label distribution shift between training and testing. Theoretically, the authors derive the generalization bound for multiclass softmax regression. They then introduce a balanced meta-softmax procedure, using a complementary meta sampler to estimate the optimal class sample rate and further improve long-tailed learning.Experiments demonstrate that this outperforms SOTA long-tailed classification solutions on both visual recognition and instance segmentation tasks. The paper was reviewed by the four reviewers that found strengths and weaknesses. The strengths were the fact that the idea is intuitive and simple to implement, the theoretical derivations in support of the method, and the good results.

Deep classifiers have achieved great success in visual recognition. However, real-world data is long-tailed by nature, leading to the mismatch between training and testing distributions. In this paper, we show that the Softmax function, though used in most classification tasks, gives a biased gradient estimation under the long-tailed setup. This paper presents Balanced Softmax, an elegant unbiased extension of Softmax, to accommodate the label distribution shift between training and testing. Theoretically, we derive the generalization bound for multiclass Softmax regression and show our loss minimizes the bound.

Deep classifiers have achieved great success in visual recognition. However, realworld data is long-tailed by nature, leading to the mismatch between training and testing distributions. In this paper, we show that the Softmax function, though used in most classification tasks, gives a biased gradient estimation under the long-tailed setup. This paper presents Balanced Softmax, an elegant unbiased extension of Softmax, to accommodate the label distribution shift between training and testing. Theoretically, we derive the generalization bound for multiclass Softmax regression and show our loss minimizes the bound. In addition, we introduce Balanced Meta-Softmax, applying a complementary Meta Sampler to estimate the optimal class sample rate and further improve long-tailed learning. In our experiments, we demonstrate that Balanced Meta-Softmax outperforms state-of-the-art long-tailed classification solutions on both visual recognition and instance segmentation tasks.